A. Field of the Invention
This invention relates generally to the field of aerial reconnaissance, remote sensing, mapping and surveillance camera systems. Generally speaking, aerial reconnaissance cameras are available in framing and pan scanning configurations, in both film and electro-optical implementations. The present invention relates to both types of camera configurations, in that a roll framing camera such as described herein generates individual frames of imagery, while the smooth rolling operation provides similar scene coverage and inertial load reductions found in pan scanning cameras.
B. Description of Related Art
In prior art framing cameras, an exposure is taken over a large scene of fixed format. The field of view of the camera is stepped across a large area using mechanically driven stepping hardware while using image motion compensation techniques to compensate for forward motion of the aircraft. The field of view of the camera is a function of lens focal length and the geometrical format size of the image recording media. The exposure time is generally controlled by a shutter and is a function of 1) the sensitivity of the photosensitive media, 2) lens transmittance and relative aperture, and 3) available scene brightness. The photosensitive material can be film, an area array Charge Coupled Device (CCD), or any other media which records an image for later retrieval.
Forward Motion Compensation (FMC) is a technique used in framing cameras to correct for the image motion on the recording media caused by forward motion of the aircraft during the exposure interval. This correction is generally introduced by moving the film or the lens to keep the image stationary in the fore/aft direction while the exposure is taking place. In a framing camera, the correction is usually accomplished by moving the film to match the rate of image motion. U.S. Pat. No. 5,668,593 to Lareau et al., assigned to the assignee of the present invention, the contents of which hare incorporated by reference herein, describes a electro-optical step frame camera system in which the forward motion compensation is achieved electronically in the focal plane of the electro-optical detector.
One limitation of a conventional film or CCD framing camera is that only a single FMC rate can be applied to any given frame regardless of the field of view. Consequently, the motion can exactly be corrected for only a portion of the image. When exposure times are short and the field angles small, this is acceptable. However, for larger fields of view and where longer exposure times are required (as at dusk or under other low light level conditions), the differential rate of motion between the film and the image increases with the field angle and can be large enough result in image blur at the edges of the field. A major advance in forward motion compensation in electro-optical framing cameras is disclosed in the Lareau et al. patent, U.S. Pat. No. 5,155,597, assigned to the assignee of the present invention. The Lareau et al. xe2x80x2597 patent, which is incorporated by reference herein, describes an electro-optical imaging array that accomplishes FMC electronically and without moving parts by dividing the columns of the array into multiple column groups, and by transferring pixel information in the column groups at a rate that substantially matches the rates of image motion in the column groups.
Another operational function of a framing camera is the generation of an overlap between successive frames of imagery. The overlap is used to ensure complete coverage of all areas of the scene, and/or to provide a view of the scene from two different angular perspectives yielding stereo imagery. In a conventional framing camera, the amount of overlap is selectable and nearly always takes place in the direction of flight.
In step frame cameras, the overlap L(OL) of the two frames of imagery is typically of 10% or 12%, or as much as 56%. An overlap of at least 50% allows all imagery in the adjacent frames to be exposed from two different angular perspectives. These images can be recombined by means of a stereo viewing system to achieve depth perception. Such stereo images are often used by a photointerpreter to gather additional information about the scene.
The operation of a film-type framing camera in a stepping mode is known in the art. For example, the article entitled xe2x80x9cThe KS-146A LOROP Camera Systemxe2x80x9d, Thomas C. Augustyn, SPIE Proceedings Vol.9, Aug. 27-28 1981, paper 309-11 p.76, describes an automatic stepping mode in which the camera cycle rate is proportional to aircraft velocity, altitude and selected depression angle, to achieve 56% overlap for stereo viewing or 12% overlap for maximum flight line coverage. With the camera line of sight normal to the flight path, the scan head provides either 1, 2, 4, or 6 lateral-step cycles. A similar stepping operation for a frame camera is described in the article entitled xe2x80x9cKS-127A Long Range Oblique Reconnaissance Camera for RF-4 Aircraftxe2x80x9d, Richard C. Ruck and Oliver J. Smith, SPIE Proceedings Vol. 242, Jul. 29-30, 1980 San Diego Paper 242-02, p.22.
Panoramic (pan) camera technology is another well-established means of imaging. In a panoramic scanning camera, the scene is exposed continuously by rotating a scanning mechanism (such as a double dove prism) so as to scan the image across the photosensitive medium. The photosensitive medium is moved in synchronism with the image. In the case of a film camera, this may be accomplished by moving the film at a constant rate past an exposure slit which is located on the lens optical axis. A scan prism located in front of the lens is rotated in synchronism with the film rate such that the image of the scene remains stationary on the film during the exposure period. The slit opening is adjusted to a predetermined width to control exposure time.
One major advantage of a pan camera is its ability to image a large area in the direction across the line of flight. Scan angles across the line of flight on the order of 120 to over 180 degrees are typical. The lens field of view in a pan camera is generally only required to be large enough to cover the width of the film. Overlapping of images and stereo imagery may also be obtained with pan cameras. Image overlap in a conventional fixed mounted pan camera is obtained as in the case of a framing camera, that is, in the common area between successive scans.
FMC for both the film and electro-optical versions of the pan camera is usually accomplished by a conventional electromechanical means. Translating the lens during the scan is a popular means to achieve graded FMC as a function of instantaneous slant range to the scene. As noted above, the FMC can be done electronically as taught in the Lareau et al. U.S. Pat. No. 5,668,593.
Prior art mechanically stepped framing panoramic cameras, such as described in the xe2x80x2593 patent and in the KS-146A camera are limited in size and the stepping rate by the mass and commensurate inertial loading created by trying to step that mass across the area of interest. Since the size and mass of the camera increases with operation in multiple spectral bands (i.e., with two or more detectors incorporated into the camera), the capability of mechanically stepped cameras is limited to smaller and more limited camera configurations.
Thus, there exists a need in the art for an electro-optical camera which obtains broad area coverage in the manner of a panning or step framing camera without the above limitations. The present invention meets that need by providing a novel roll framing technique for generating broad area coverage with an area array image recording medium, described in more detail herein. The image motion due to camera roll is compensated for electronically in the detector array. The invention is also particularly suitable for larger, more massive, and more complex cameras, including a camera which carries two or more imaging detectors in order to generate frames of imagery in two or more different bands of the electromagnetic spectrum simultaneously.
The present invention provides the capability for collection of imagery using a framing camera in which a continuous scan motion about the roll axis of the aerial reconnaissance vehicle is performed, a technique referred to herein as xe2x80x9croll framingxe2x80x9d. As the camera rototes about the roll axis in a continuous fashion, the roll motion is compensated for electronically. This enables high resolution imagery to be generated without loss of resolution or blur, due to the fact that relative motion of the image with respect to the image recording media caused by the roll motion is compensated for using the techniques described herein.
The continuous roll motion of the camera facilitates image collection without large inertial accelerations and decelerations or large power spikes, as are found in prior art step frame camera system when the camera mass is physically stepped across the terrain of interest in a series of start and stop movements. The present invention is believed superior to prior art step framing cameras since the problems inherent with mechanical stepping are eliminated. The camera and method are applicable to all sizes and arrangements of cameras, including cameras implementing single spectrum, multi-spectrum and hyperspectral optical systems. The invention is also applicable to cameras with mechanical shutters, electronic shutters, acoustical/optical switches, and other electronic exposure controls.
Thus, in a first aspect of the invention, a method is provided for imaging a scene with a framing camera installed in an aerial reconnaissance vehicle. The camera comprises a two dimensional array of photosensitive cells, an optical system directing scene radiation onto said array, and a mechanism for rolling the camera about a rotation axis. The array of cells store pixel information and is arranged in a plurality of rows and columns. The method comprises the steps of:
(a) continuously rotating the camera about the rotation axis with the roll mechanism to thereby direct scene information onto the two dimensional array;
(b) exposing the array while the camera is rotating and transferring pixel information in the array at a rate substantially equal to an image motion rate due to the rotation of the camera;
(c) reading out the pixel information from the array; and
(d) repeating said steps (b), and (c) while the vehicle flies past a scene of interest and while the camera continuously rotates about the roll axis, thus generating a series of frames of imagery.
In a preferred embodiment, the camera is mounted to the aerial reconnaissance vehicle such that the step of continuously rotating comprises the step of rotating said camera about an axis substantially parallel to the direction of forward motion of the reconnaissance vehicle. However, the camera could also be mounted in an orthogonal configuration such that the step of continuously rotating comprises the step of rotating the camera about an axis in a direction substantially orthogonal to the direction of forward motion of said aerial reconnaissance vehicle. In this less preferred embodiment, the camera could roll essentially about the pitch axis and generate a series of images in the forward oblique direction towards nadir.
In a typical embodiment, the steps (a), (b), (c), and (d) recited above are performed in a a series of cycles as the aircraft flies past a scene of interest. The frames overlap one another so as to avoid gaps in scene coverage. If the overlap is sufficient, it would be possible to obtain stereo imagery of the scene of interest. The camera can be configured with just a single detector and generate imagery in a single band of the electro-magnetic spectrum. Alternatively, the camera includes a second electro-optical detector and the camera generates imagery in two bands of the electromagnetic spectrum simultaneously from the first and second detectors. The preferred embodiment described in detail herein is an example of a dual band imaging system. As yet another alternative embodiment, the camera includes an electro-optical detector and optical system for generating imagery in a pan-chromatic spectral band, such as a hyperspectral electro-optical imaging array.
In another aspect of the invention, an electro-optical roll framing camera with electronic roll motion compensation is provided. The camera is adapted for installation in an aerial reconnaissance vehicle. The camera comprises an electro-optical detector comprising a two-dimensional array of photosensitive cells that store pixel information. The array is arranged in a plurality of rows and columns and has at least one readout register for reading out pixel information from the array. The camera further includes an optical system directing scene radiation onto the array. A servo-mechanical system is provide which couples the camera to the aerial reconnaissance vehicle which is adapted or configured for continuously rolling the camera about a rotation axis to thereby direct scene information onto the optical system and array. Further, roll motion compensation circuitry is provided for electronically transferring pixel information in the array of photosensitive cells at a rate substantially matching the rate of image motion due to the rotation of the camera, whereby the resolution of images generated by said array may be preserved.
In the illustrated embodiment, the servo-mechanical subsystem includes a first motor system coupled to the camera housing that rotates the camera housing (including the optical system as recited above) about a first axis. The camera housing is preferably installed in the aerial reconnaissance vehicle such that this first axis of rotation is parallel to the roll axis of the aerial reconnaissance vehicle (referred to herein for simplicity as xe2x80x9cthe roll axisxe2x80x9d). The image recording media are exposed to the scene to generate frames of imagery as the first motor system rotates the camera housing in a continuous fashion about the roll axis. The first and second image recording media have a means for compensating for image motion due to the rotation of the camera housing. In an electro-optical embodiment of the image recording media, the roll motion compensation means is preferably comprised of electronic circuitry for clocking or transferring pixel information through the electro-optical detectors at a uniform rate substantially equal to the rate of image motion due to camera rotation. A method of calculating the image motion rate, and thus pixel information transfer rate, due to roll of the camera housing is disclosed herein. If a film camera is used for the image recording media, a mechanical system is used to move the film at a rate substantially equal to the image motion rate.
In the preferred embodiment, the servo-mechanical subsystem also includes a second motor system coupled to the Cassegrain optical system. The second motor system rotates the Cassegrain optical system about a second axis in the direction of forward motion of the reconnaissance vehicle to compensate for forward motion of the aerial reconnaissance vehicle. The action of the first motor assembly to rotate the entire camera housing about the roll axis occurs at the same time (i.e., simultaneously with) the action of the second motor system to rotate the Cassegrain optical system in the line of flight to accomplish forward motion compensation. The net effect of the action of the second motor system and the roll motion compensation system is that the image of the scene of interest is essentially frozen in the focal plane while the image recording media obtain the frames of imagery, allowing high resolution images of the scene to be obtained.
In a preferred embodiment, the camera is a dual band framing camera, and there are first and second image recording media each comprising two dimensional area array electro-optical detectors. One may be manufactured from materials sensitive to radiation in the visible portion of the electromagnetic spectrum, and in a preferred embodiment is a charge-coupled device (CCD) detector of say 5,000xc3x975,000 or 9,000xc3x979,000 pixels. The other of the electro-optical detectors is made from a material sensitive to radiation in the infrared portion of the electromagnetic spectrum, and may be a platinum silicide array of photo diode detectors or other suitable type of electro-optical detector suitable for IR detection. The detector sensitive to radiation in the infrared portion of the electromagnetic spectrum is preferably sensitive to radiation having a wavelength of between 3.0 and 5.0 microns (MWIR), or from about 8.0 to about 14.0 microns (LWIR). In either of the embodiment of electro-optical detectors, they will typically comprise an array of pixel elements arranged in a plurality of rows and columns. The means for compensation for roll motion of the camera housing comprises electronic circuitry for transferring pixel information in the electro-optical detectors from row to adjacent row at a pixel information transfer rate (uniform across the array) substantially equal to the rate of image motion in the plane of the electro-optical detectors due to roll of the camera housing. Thus, the roll motion compensation can be performed electronically on-chip.
As a further possible embodiment, electro-optical detectors with the capability for transferring pixel information in both row and column directions independently, such as described in Lareau et al., U.S. Pat. No. 5,798,786, could be used for the image recording media. Forward motion compensation and roll motion compensation could be performed on-chip in the detectors.
The present invention required the solution to several difficult technical challenges, including optical, servo-mechanical and operational difficulties. For an electro-optical framing LOROP camera to operate in a continuous sweep with a framing array with at least two discrete bands of the electromagnetic spectrum at the same time, the challenge is to accurately compensate for the roll motion electronically at a focal plane detector with (1) good image quality and satisfactory modulation transfer function, (2) while minimizing inertial loading, and (4) enabling the use of a relatively large two-dimensional area array as a focal plane detector to get an adequate field of view and resolution. In accordance with one aspect of the invention, these optical challenges were solved by an on-chip roll motion compensation described in more detail herein.
The inventive multi-band LOROP/Tactical camera using electronic roll motion compensation does not lend itself to the use of servo-mechanical systems developed for prior art LOROP systems, particularly those used in prior art step frame cameras (such as described in the Lareau et al. xe2x80x2593 patent). The prior art step frame cameras use a stepping mirror to step across the line of flight and direct radiation onto the array, and require a de-rotation mechanism, such as a Pechan prism, to de-rotate the images. The standard solution of stepping the entire LOROP camera system or even a large scan mirror assembly at the operational frame rate are not acceptable alternatives for large LOROP cameras, and in particular large dual band systems. In particular, the applications of the present invention are flexible enough to include both strategic and tactical aircraft, as well as the new breed of aircraft being used by the military for reconnaissance known as unmanned aerial vehicles (including low observables). The diversity of these applications posed a power and stability problem that prevents application of prior art solutions. The task of stepping a 400 lb. camera mass two to four times a second creates tremendous inertial loads as well as power spikes that would be unacceptable. Even the inertia and associated settling times of a stepped scan head assembly pose problems in some applications.
This servo-mechanical situation required a unique inventive solution, described in detail herein. The solution, as provided in one aspect of the present invention, was to (1) rotate the entire camera (including the entire optical system and the image recording media) smoothly in a continuous fashion about an axis parallel to the aircraft roll axis, similar to the pan-type movement, but without the starts and stops used in a traditional step-frame camera system, and (2) operating the camera as a framing camera while the camera undergoes the smooth rotation. Frames of imagery are thus taken while the camera smoothly rotates about the roll axis at a constant angular velocity. In addition to this novel xe2x80x9croll-framingxe2x80x9d technique, the present invention also electronically compensates for, i.e., stops, the image motion due to roll while the camera is scanning in a smooth motion. Meanwhile, a novel forward motion compensation technique is performed by the Cassegrain optical assembly to cancel out image motion effects due to the forward motion of the aircraft. The result enables exposures of the image recording media to the scene while compensating for roll and forward motion, enabling high-resolution images to be obtained.
The present invention thus solves the difficult optical, servo-mechanical and operational problems and provides a dual band framing electro-optical LOROP camera that delivers a performance and technical capability that has never before been achieved. In particular, it provides a system by which high-resolution frames of imagery in two different portions of the electromagnetic spectrum can be generated simultaneously. The inventive camera can be used in a quasi-stepping mode, in which overlapping frames of imagery are obtained across the line of flight. It can also be used in a spot mode, in which the camera is oriented in a particular direction to take an image of a target expected to be in the field of view.
Many of the teachings of the present invention are particularly applicable to a dual band electro-optical framing reconnaissance camera, and such a camera is the preferred embodiment. However, as explained below, some of the techniques and methods of the subject camera system, such as the roll-framing operation and unique roll and forward motion compensation techniques, are applicable to a camera system that images terrain in only one portion of the electromagnetic spectrum. Thus, in an alternative embodiment the camera is basically as set forth as described above, except that only a single detector is used and a spectrum-dividing prism and second optical path are not needed. Furthermore, while a preferred embodiment uses a two-dimensional electro-optical imaging array for the detector in each of the bands of the electromagnetic spectrum, the inventive camera system can be adapted to use film or other types of detectors for the photosensitive recording medium. In the film camera embodiment, roll motion compensation could be performed by moving the film in a manner such that the film velocity substantially matches the image velocity due to camera roll.
While the foregoing summary has described some of the highlights of the inventive camera system, further details on these and other features will be described in the following detailed description of a presently preferred embodiment of the invention.